
Teachers and Curriculum 


ISSN 2382-0349 
Website: http://tandc.ac.nz 



Wilf Malcolm Institute 
of Educational Research 

Te Putahi Rangahau Matauranga o Wilf Malcolm 


THE UNIVERSITY OF WAIKATO 


Title of Issue/Section Teachers and Curriulum, Volume 17, Issue 2, 2017. Special 
Issue: Mobile Technologies and Learning 

Editor/s Nigel Calder and Carol Murphy 


To cite this article: Jowsey, L., & Aguayo, C. (2017). Te reo Maori ka rere: ‘Talknology’ and Maori 
language as a language of choice. Teachers and Curriculum, 77(2), 95-102. 
http://dx.doi.org/10.15663/tandc.vl7i2.166 

To link to this article: http://dx.doi.org/10.15663/tandc.vl7i2.166 

To link to this volume http://dx.doi.org/10.15663/tandc.vl7i2 


Copyright of articles 

Creative commons license: https://creativecommons.Org/licenses/by-nc-sa/3.0/ 

Authors retain copyright of their publications. 

Author and users are free to: 

• Share —copy and redistribute the material in any medium or format 

• Adapt —remix, transform, and build upon the material 

The licensor cannot revoke these freedoms as long as you follow the license terms. 

• Attribution —You must give appropriate credit, provide a link to the license, and indicate if changes were 
made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or 
your use 

• Noncommercial —You may not use the material for commercial purposes. 

• ShareAlike —If you remix, transform, or build upon the material, you must distribute your contributions under 
the same license as the original. 

Terms and conditions of use 

For full terms and conditions of use: http://tandc.ac.nz/tandc/about/editorialPolicies#openAccessPolicy 














O-T0-KAPUA ("WHAT CLOUDS SEE 7 ): A MIXED REALITY EXPERIENCE BRIDGING 
ART, SCIENCE AND TECHNOLOGY IN MEANINGFUL WAYS 


SUSAN JOWSEY AND CLAUDIO AGUAYO 

Auckland University of Technology 
New Zealand 

Abstract 

Mixed Reality learning environments can provide opportunities to educationally enhance previously 
isolated scientific concepts by using art and technology as mediums for understanding the world. 
Participatory experiences provide a kinetic means of comprehending often-abstract knowledge, 
creating the conditions for sensory learning that is inclusive and accessible. The O-Tu-Kapua (what 
clouds see) Mixed Reality experience provides an inverted view of the environment. Rather than us 
looking up towards the sky, the clouds look down upon us. Atmospheric science is personified through 
an intertwined narrative that begins with creative action. Participatory learning embedded within 
technology-enhanced education creates opportunities to question how as educators we can create 
opportunities for integrating art, science and technology, enhancing our ability to capture young 
people in conversations about the socio-ecological Anthropocene. 
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Introduction 

This project was part of a larger climate change initiative known as TEMP (see 
http://www.tempauckland.org.nz) , focusing on climate change in the era of the Anthropocene (Taylor 
& Pacini-Ketchabaw, 2015). Five teams of artist(s) and scientist(s) were invited to collaborate in the 
creation of high impact participatory experiences presented as a series of events and installations 
during March to May 2017 within two hubs: Corban Estate Arts Centre in Henderson, and Te Uru 
Waitakere Contemporary Gallery in Titirangi. O-Tu-Kapua (What clouds see) was the multimodal 
creative learning environment produced by the ‘TEMP Air’ team, who were tasked with raising 
awareness about the importance of air and air quality. The team consisted of F4 Collective, 
atmospheric scientists from NIWA (National Institute of Water and Atmospheric Research), 
researchers from Auckland University of Technology and a private sector Augmented and Virtual 
Reality company. 

The TEMP Air team researched and developed strategies over a two-year period that would 
creatively introduce a range of science concepts to west Auckland tamariki. Loosely building on the 
New Zealand Level 5 science curriculum, Making Sense of the Living World (Ministry of Education, 
2007), the team investigated how air quality might be incorporated into a holistic conversation about 
our interrelatedness with the environment. Integral to this was questioning how the project could 
combine analogue-making with interactivity and mobile technology. Employing a sensory 
methodology (Pink, 2015), the team focused on developing participatory multisensory experiences 
that could spark conversation and curiosity to foster learning about birds and insects that live in New 
Zealand’s native forests, the life of the trees and by necessity coming to understand our shared 
dependency on air. 

O-Tu-Kapua combined the personal and imaginative haptic world of the child through their individual 
artistic efforts, with a place-based educative narrative (Penetito, 2009), linking their creative 
endeavours to scientific stories about the flora and fauna sharing their local habitat. Embodying the 
living world, their artworks flew and perched amongst the branches of the fictive forest creating an 
immersive and engaging visual world, a vibrant habitat brought to life through their collective efforts. 
Hidden amongst this profusion were Augmented Reality (AR) markers disguised as drawings. 
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Discovering them allowed tamariki to perceive the secret life of the forest. The O-Tu-Kapua app 
extended the sensorial through composed sounds, representations of the plunking rain, the call of the 
birds, and the music of the Hauauru, the west wind, as he plays music with the plants and trees. The 
health of our atmosphere and the trees were pictorially embodied, each element revealed through a 
digital extension of the haptic. Technology and analogue making conjoined to create an experiential 
sensory investigation of the living world. 

Designing O-Tu-Kapua 

From the outset O-Tu-Kapua was designed to engage tamariki (primarily focusing on 8 to 10 years of 
age) through activation of the senses, and to create meaningful connections (Thompson, 2002; Tilbury 
& Wortman, 2008). Rooting the project in the local, in this instance, the Waitakere Ranges (the 
largest forest present in West Auckland) provided an anchor, enabling the connection of basic 
concepts of atmospheric science with more integrated concepts and notions of ecological thinking and 
literacy (Orr, 1992; Sterling, 2005). Coupling a Place-Based Educational framework with a sensory 
methodology, the team aimed to create a pedagogical agent capable of being activated in differing 
circumstances and through different means (analogue hands-on, aural, digital, in-context and 
connected modes of learning). A Mixed Reality quest-based experience was used to augment learners’ 
meaning-making by enabling participation in media-rich environments (Pachler, Bachmair, & Cook, 
2010). Learners were engaged in perceptual exploration and contributed to the creation of a forest-like 
metaphorical environment (through school workshops). They investigated and located hidden worlds 
within the forest (with and without the assistance of mobile devices), all the while seeing and 
perceiving the world they inhabit and that inhabits them through situated and embodied science. In 
this relational world, curiosity and pondering are concomitant with wonder. 

Throughout the nearly two-year development of the project, O-Tu-Kapua engaged eight key 
individuals, most of who work in the tertiary sector, as well as an industry partner and a small group 
of postgraduate students. Everyone who worked on this project was motivated by the belief that 
tamariki from New Zealand, and across the globe, will increasingly be faced with environmental and 
societal issues related to the earth's changing climate. During the planning of O-Tu-Kapua the team 
grappled with widely held beliefs that young people were not able to engage with climate science 
concepts. Any form of public discussions around the mechanisms, impacts and options for adaptive 
behaviours predominantly exclude the input of tamariki. However, the research team agreed it was 
essential to foster creative and constructive conversations about air and air quality, especially amongst 
the young, if we are to develop an interrelated conception of environment. 

Technology-enhanced learning in practice: The O-Tu-Kapua Mixed Reality experience 

Whilst delivered through the mechanisms provided by being an art exhibition, O-Tu-Kapua remained 
steadfastly an educational endeavour. Developing a Mixed Reality (MR) environment (FitzGerald et 
al., 2013) required a framework that supported movement between the analogue and digital 
experiences, which encompassed the real environment (present at the gallery space), integrated with a 
set of digital affordances (‘possibilities’ offered by digital tools). The digital components of the MR 
environment included Augmented Reality (AR), aural soundscapes and an app-based 360 degree 
panoramic tour of the Waitakere Range that could be viewed in Virtual Reality (VR) mode through 
cardboard VR headsets (provided as part of the installation). The MR was complemented by a website 
supporting extended inquiry through the provisioning of links to a large variety of online resources 
and social media (see https://www.otukapua.nz/) , enhancing the emphasis in O-Tu-Kapua on sharing 
and connecting. MR environments fundamentally blend paradigms, providing a perfect context for 
developing an integrated multimodal approach. In this instance, the tamariki were pedagogically 
supported to contribute, through art-making (Figure 2), to the construction of a real environment; one 
they could walk around in and identify their contribution and those of their peers. They could show 
their whanau their artwork and speak through their newly acquired knowledge about the birds, insects 
and trees and our shared need for good quality air. 

Though constructed in a gallery (Figure 3), this experience could, and has been, reconstructed in a 
classroom. Supporting this physical reality is a secondary experience that is accessed through mobile 
technology, iPads, tablets or smartphones. The app magnifies the experience and extends the narrative. 
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Housed inside the app is a hand-drawn magnifying glass (Figure 1), which is used to locate the AR 
component of the MR environment, mirroring a real world scientific investigative technique. With 
MR, as opposed to Virtual Reality or app-based games, the mobile device can operate as a social 
agent. Participants are not disconnected from the collective experience, as they would be if they 
required a VR headset or were connected to the world of the game. The AR enhances collective 
wonder because it is situated and embodied in the real. It is this intertwining that promotes face to 
face communication and discussions; together the analogue and digital manifestations of reality 
orchestrate shared experiences and thus social learning. 

By combining digital and analogue stimuli, O-Tu-Kapua aimed to promote exploration through 
individual and group-based active learning. Using markers (Figure 4) to create activation points 
readable by the view of the camera on the mobile device, much like a QR code, a layer of digital 
information (e.g., animation, sound and data visualisations) was added to the physical, sensory 
worldview. MR allows for the integration of real and digital information as a combined experience 
(FitzGerald et al., 2013). Motivated by encouraging tamariki to pose and seek to answer questions, we 
shaped the exhibition to represent a self-determined learning space. Tamariki and gallery visitors were 
able to create their own learning content and context (Aguayo, Cochrane, & Narayan, in press; Hase 
& Kenyon, 2013) with or without the assistance of mobile devices. Loosely orchestrated through a 
quest concept, additionally, the team provided students and the general public with a folded, double 
sided A4 booklet as an entree to the quest (Figure 5). This approach stimulated physical exploration 
of the environment (the forest setting constructed in a gallery) as well as through technology, creating 
an ecology of resources (EOR) (Luckin, 2008) and facilitating an integrated technology-enhanced 
learning environment. A hand drawn magnifying glass (Figure 1), present throughout the experience 
and built into the O-Tu-Kapua app (https://www.otukapua.nz/the-app) , reinforced the notion of 
looking thoughtfully and with purpose. 



Figure 1: Representation of the quest concept through the use of a magnifying glass 
(source: S. Jowsey). 

O-Tu-kapua’s meta-narrative was underscored by the project’s sensory methodology. Collectively we 
compose, support or destroy the ecosystem we inhabit. By making the birds, insects and leaves for the 
trees, the tamariki gain awareness. Individually and as a whole they were instrumental in enabling the 
habitat to fill with life. Through the app, life gained additional sensory dimensions, sound and 
movement, whilst every aspect of the MR experience responded to touch. The majority of AR 
markers triggered aural soundscapes. Enlivening an otherwise silent visual world the sounds created 
by Maori composer Maree Sheehan, used contemporary instruments, taonga puoro (traditional Maori 
instruments) and voice, fused Western scientific concepts with Maori cosmologies. The forest is an 
ecosystem essential to maintaining life, and a sentient being, an embodiment of the gods, in this 
instance Hauauru. 

The aural dimension manifests the incorporeal, where the evocative coalesces with familiar sounds: 
the wind, the rain, the songs of birds and insects. Making visible the invisible, linking the socio- 
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cultural space of learning (what is being taught) with the personal space of the learner (the learner’s 
existing knowledge, either learnt or experienced) (Aguayo, 2016). Predicated on the belief that there 
are different ways of explaining the world we live in, O-Tu-Kapua demonstrates that what may be 
perceived as ordinary is in fact extraordinary. 



Figure 2: School hands-on workshop where students made bugs and birds to go in the 
forest environment (source: Unlocking Curious Minds, 2017 (left) and Te Uru 
Waitakere Contemporary Gallery Educational Team (right)). 



Figure 3: O-Tu-Kapua installation and exhibition space. Workshop bugs and birds were 
continuously added to this evolving learning space (Source: Te Uru Waitakere 
Contemporary Gallery Education Team). 
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Figure 4: Tree stump AR marker (source: F4 Collective © - left). The triggered AR tree 

visible using the O-Tu-Kapua app (source: F4 Collective - right) shows real-time 
weather and traffic conditions in Auckland. Try it yourself by downloading the 
app (freely from app stores for iOS and Android devices). 



If you would Ilk© to download the augmented reality app to help you 
in your quest, you can find it in the Google Play store or Apple App store. 
Search for O Tu Kapua 


Or you can use the QR code on the wall of the gallery to find it. 


Did you know that the water that rains down now is the same water that 
rained down when the moa walked amongst the forests of Aotearoa? 


0 Z /What kind of molecule 


am I ? 



mlocd or us© our app on your device, tr 
30 to our website www.otukapua.nz 
ormatlon to help you complete the quest 


Warm air is less dense than cold air, so hot air 
can hold a lot of water vapour, which is 
why in summer it often feels so sticky 


M IF £s 

(SviMfywIhasm 


Did you know that when you 
breathe out you release water 
vapour. Plants do this too, 
it's called transpiration. 


but air is invisible. 
However, you can feel tf 
Wind, humidity, and you 
can see the clouds and/* 
capture the rain! 


Trees use transpiration to control the 
temperature of the air around them, 
leaves play an important part in this 
process too. 


Rain is part of the Water Cycle... about 
70% of the earth is covered with water. 


Humidity is. 


The clouds produce rain (precipitation), which 
falls to the ground, giving the trees something to 
drink. ..then the water evaporates back up into 
the sky, making more clouds (condensation). 

This is the Water Cycle. 


Water molecule 


Air Molecu 


Figure 5: The A4 folded booklet (source: F4 Collective ©) download from the O-Tu- 
Kapua website. 

Whilst each aspect of the MR experience was interwoven conceptually and aesthetically, the main 
strands were designed so that they could exist, and remain effective, independently. Visitors to the 
exhibition did not have to attend the workshop to understand and explore the exhibition, or need the 
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app in order to be intrigued and curious about the quest. Employing pedagogical link-making that 
focuses on how “connections between ideas in the ongoing meaning-making interactions of classroom 
teaching and learning occurs” (Scott, Mortimer. & Ametller, 2011, p. 3), the team sought to provide a 
multimodal journey. Utilising creative thinking, the TEMP Air team explored principles such as 
predict, observe and explain, illustrate ideas, experiment, collaborate and reflect (Linn, Bell, & Hsi, 
1998; Mehlenbacher, 2010) infusing them into the design of the project. 

Learning outcomes 

O-Tu-Kapua staged the asking of speculative questions; for example, a marker that depicted a tree 
stump, when viewed through the app, showed a tree. The health of which responded to real time data 
pertaining to Auckland weather and traffic conditions. This data stream was collected from various 
sources and refreshed every 10 minutes (Figure 4). The responsive nature of the tree set the stage for a 
host of questions and hypotheses about what was happening to, and within, the immediate 
environment of the tamariki. Whilst you can look out a window or stand outside and make predictions 
about the weather, it is hard to envision how much pollution is in the air, unless you can see or smell 
car exhaust or wood smoke. Creating pedagogical links between curricula and lived experience 
frames interconnections from which larger conceptual webs or patterns can be formulated (Scott, 
Mortimer et al., 2011; Sterling, 2005). 

This form of mediated learning can sustain a longer-term engagement with the subject; abstract 
scientific notions concretise through associative thinking. A simple concept such as air is all around 
me including inside me, can change the way air is perceived. Air is not an abstract blue field viewed 
as the sky, it fills my lungs and those of the birds in the trees outside. Threaded through O-Tu-Kapua 
are familiar and unfamiliar notions of air (Figure 6). Exploring the world through a microscope 
reveals the unseen, the human propensity to wonder and to imagine. To be curious has provided the 
impetus to develop technologies that reveal, if not necessarily explain what is hidden from the human 
eye. MR offers a mechanism for combining established teaching practices with technology based 
experiences in transformative ways (Aguayo et al., in press), where AR is augmentation of reality that 
is constructed , created by the tamariki as part of their learning experience. 



Figure 6: A primary school class engaging with the exhibition space (Laura 
Goodall/Curious Minds NZ, 2017). 
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Concluding remarks 

The creation of O-Tu-Kapua taught the research team a lot about how Mixed Reality environments 
can support a deeper connection to scientific concepts, not just for tamariki. The project received 
positive responses from participating schools as well as the wider community. Feedback from Te Urn 
Waitakere Contemporary Gallery stated that “people of all ages and abilities have engaged with this 
exhibition. From 2 year-olds spending an hour looking at the bugs and learning about the forest while 
enjoying the app, through to elderly people who have embraced the technology and appreciated the 
stunning children’s artwork. Home school groups have spent long periods of time in the gallery, 
sketching trees and using the app to develop further understanding on our local environment and 
climate science”. Fascination and curiosity is not the sole domain of the young. We should all be 
allowed to marvel, not at technology but at the wonder of the world we inhabit. Mixing realities can 
prise open the human capacity for sensory-based inquiry, simultaneously feeding our hunger for 
embodied, emplaced understanding. 

This project raises many questions about how learning opens opportunities for integrating artistic 
expression and narrative with scientific concepts, the role of technology, and our ability to capture 
young people in conversations about the Anthropocene and our engagement with the invisible. 
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Glossary of te reo Maori words 

O-Tu-Kapua What Clouds See 

Tamariki Children 

Kaiako Teacher 

Whanau Family 

Taonga puoro Traditional Maori instruments 

Hauauru is the playful west wind 
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